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GAO 


United  States 

General  Accounting  Offlce 

Washington,  D.C.  20548 

Information  Management  and 
Technology  Division 

B-220361 

January  18, 1989 

The  Honorable  Dorcas  R.  Hardy 
Commissioner  of  Social  Security 

Dear  Ms.  Hardy:  ^ 

/  ' 

This  report  discusses  our  review  of  the  Social  Security  Administration’s  computer  capacity 
management  program.  We  reviewed  this  pi^gram  under  our  basic  legislative  authority  to 
evaluate  federal  agencies  and  programs.  Our  review  assessed  the  Social  Security 
Administration’s  activities  for  managing  existing  computer  resources  and  planning  for  future 
resource  needs. 

This  report  contains  recommendations  to  you  in  chapter  4.  As  you  know,  31  U.S.C.  720 
requires  the  head  of  a  federal  agency  to  submit  a  written  statement  of  actions  taken  on  our 
recommendations  to  the  House  Committee  on  Government  Operations  and  the  Senate 
Committee  on  Governmental  Affairs  not  later  than  60  days  after  the  date  of  this  report  and 
to  the  House  and  Senate  Committees  on  Appropriations  with  the  agency’s  first  request  for 
appropriations  made  more  than  60  days  after  the  date  of  this  report.  We  would  appreciate 
receiving  copies  of  these  statements. 

We  are  sending  copies  of  this  report  to  the  Chairmen  of  the  above  Conunittees;  the  Director, 
Office  of  Management  and  Budget;  and  the  Administrator,  General  Services  Administration. 
This  report  was  prepared  under  the  direction  of  Melroy  D.  Quasney,  Associate  Director. 

Other  major  contributors  are  listed  in  appendix  11. 


Sincerely  yours. 


Ralph  V.  Carlone 
Assistant  Comptroller  General 
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Elxecutive  Summary 


Purpose 

_^;-,The  Social  Security  Administration  (ssa)  depends  heavily  on  computers 
to  perform  its  mission,  which  includes  maintaining  records  on  over 

300  million  people  and  paying  in  excess  of  $230  billion  annually  in  bene¬ 
fits  to  over  40  million  people.  Since  1982,  the  agency  has  estimated  it 
has  spent  approximately  $643  million  for  its  computer  systems  and  is 
planning  to  spend  about  $i7u  million  in  fiscal  year  1989  to  maintain  and 
further  modernize  its  systems. 

'  Because  of  the  considerable  investment  in  the  agency’s  computer  sys¬ 
tems  and  their  importance  to  the  agency’s  mission,  gao  reviewed  ssa’s 
computer  capacity  management  activities  to  evaluate  the  effectiveness 
of  ssa’s  activities  for  managing  existing  computer  resources  and  plan¬ 
ning  for  future  resource  needs.  . 

Background 

In  1982.  ssa  proposed  a  comprehensive  5-year  modernization  plan  for  all 
of  its  computer  systems  that  included  the  purchase  and  installation  of 
new  high-speed  mainframe  computers  and  associated  hardware  to, 
among  other  things,  expand  and  modernize  the  processing  capabilities  of 
its  field  offices.  In  June  1987,  ssa  purchased  three  mainframe  computers 
to  support  redesigned  software  programs  and  other  field  office  system 
upgrades.  The  agency  spent  $5.1  million  in  September  1988  to  increase 
its  computer  capacity,  and  is  now  planning  additional  computer  procure¬ 
ments  for  1989  and  perhaps  1990  to  support  the  field  office  system. 

Federal  Information  Resources  Management  Regulations  require  govern¬ 
ment  agencies  to  conduct  capacity  management  activities  in  planning  for 
and  acquiring  computer  resources.  Such  activities  are  important  because 
they  provide  agencies  with  information  about  1)  the  computer  capacity 
current  operations  are  using  and  2)  the  additional  computer  capacity 
that  will  be  required  to  support  increased  automation.  Capacity  manage¬ 
ment  activities  include  collecting  and  analyzing  detailed  performance 
data  on  current  computer  processing,  and  comprehensive  modeling  and 
pilot  testing  of  planned  computer  systems. 

Results  in  Brief 


GAO  found  that.  SSA  has  not  effectively  performed  capacity  management 
activities  needed  to  accurately  assess  its  computer  capacity  needs.  While 
the  agency  has  collected  some  data  and  performed  some  systems  model¬ 
ing  and  pilot  testing,  these  activities  have  not  been  adequately  per¬ 
formed  to  enable  SSA  to  determine  computer  capacity  requirements  with 
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reasonable  assurance.  Further,  an  ssa  internal  study  team  and  two  con¬ 
tractors  reported  during  the  past  year  that  the  agency's  capacity  man¬ 
agement  program  needed  improvement  in  many  areas. 

SSA  has  initiated  actions  to  improve  its  capacity  management  program 
and  expects  to  complete  these  actions  by  the  end  of  1989.  While  the 
agency’s  actions  are  noteworthy,  until  further  capacity  management 
improvements  are  completed,  ssa  will  not  know  with  assurance  how 
much  computer  capacity  is  required  to  support  its  operations — informa¬ 
tion  it  needs  to  effectively  justify  future  computer  procurements. 


Principal  Findings 


Collection  and  Analysis  of 
Capacity  Management 
Information  Needs 
Improvement 


Federal  Information  Resources  Management  Regulation  201-30  (Manage¬ 
ment  of  ADP  Resources)  requires  the  routine  collection  and  analysis  of 
detailed  capacity  management  data  to  measure  current  computer  use 
and  needs,  and  to  predict  future  capacity  requirements,  ssa  does  not  sys¬ 
tematically  collect  and  analyze  sufficiently  detailed  data  to  effectively 
perform  these  functions.  For  instance,  while  the  agency  prepares  vari¬ 
ous  reports  on  total  system  utilization,  these  reports  do  not  have  suffi¬ 
cient  detail  at  the  transaction  level  that  SvSA  needs  to  effectively 
determine  whether  the  current  computer  processing  environment  is 
operating  effectively  and  efficiently.  (See  pp.  14  and  15.) 


Data  Limitations  Affecting  Modeling,  a  technique  for  simulating  computer  system  performance 
SSA’s  Modeling  Efforts  under  various  assumptions,  can  be  used  to  help  satisfy  federal  regula¬ 

tion  requirements  for  analyzing  current  computer  performance  and  pre¬ 
dicting  future  capacity  requirements.  Inadequate  data  collection 
(1)  during  new  software  development  and  pilot  testing  and  (2)  from  cur¬ 
rent  computer  utilization  have  adversely  affected  the  reliability  of  ssa’s 
modeling  efforts.  For  example,  one  of  the  agency's  modeling  efforts  indi¬ 
cates  that  the  capacity  of  the  current  computer  system  used  to  support 
the  field  offices  may  be  exceeded  in  1988  in  the  worst  case  scenario,  or, 
in  the  best  case  scenario,  may  be  sufficient  until  1992.  The  disparity 
between  these  results  raises  questions  as  to  when  and  how  much  addi¬ 
tional  computer  capacity  is  required  to  meet  ssa's  needs.  (See  p.  17.) 
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Independent  Assessments 
Find  Capacity 
Management  Activities 
Need  Improvement  to 
Better  Justify  Future 
Procurements 


In  May  1987,  approximately  5  months  after  we  began  this  review,  ssa 
formed  a  capacity  modeling  and  planning  team  to  evaluate  and  improve 
its  capacity  management  program.  In  1988,  the  agency  also  had  two  con¬ 
tractors  perform  independent  assessments  of  the  capacity  management 
program,  one  as  part  of  a  larger  study  of  ssa’s  integration  planning  pro¬ 
cess,  the  other  specifically  related  to  the  agency’s  capacity  management 
process.  The  evaluation  specifically  relating  to  ssa’s  capacity  manage¬ 
ment  process  recognized  the  maturity  and  comprehensiveness  of  the 
process  the  agency  was  implementing  and  reported  that  the  process  as  it 
now  stands  is  producing  some  valuable  and  directly  useful  information. 
However,  all  three  evaluations  raised  questions  about  ssa's  ability  to 
reasonably  determine  the  agency’s  current  and  future  computer  capac¬ 
ity  requirements  and  recommended  actions  designed  to  improve  the 
agency’s  data  gathering  and  modeling  activities,  and  its  overall  manage¬ 
ment  of  the  capacity  planning  and  procurement  processes.  (See  pp.  20  to 
23.) 


Efforts  to  Improve 
Capacity  Management  Not 
Yet  Complete 


The  agency  has  implemented  some  of  the  recommended  corrective 
actions,  such  as  better  defining  the  capacity  management  program,  and 
establishing  user  service  agreements  for  its  on-line  system,  ssa  also  has  a 
schedule  for  further  improving  its  capacity  management  activities,  and 
estimates  that  these  improvements  will  take  about  a  year  to  implement. 
Until  improvements  to  the  capacity  management  program  are  com¬ 
pleted,  SSA  will  not  be  able  to  effectively  determine  its  ongoing  capacity 
requirements,  and  therefore  provide  assurance  that  its  future  computer 
acquisitions  are  justified.  (See  p.  23.) 


Recommendations  to 
the  Secretary  of 
Health  and  Human 
Services 


GAO  recommends  that  the  Secretary  of  Health  and  Human  Services 
direct  the  Commissioner  of  Social  Security  to: 

ensure  that  ssa  improves  its  capacity  management  program  as  currently 
scheduled  and  uses  this  program  to  justify  future  acquisitions, 
avoid  executing  major  computer  acquisitions  until  an  effective  capacity 
management  program  is  in  place.  If  ssa  believes  that  computer  resources 
must  be  acquired  before  an  effective  capacity  management  program  is 
in  place,  it  should  demonstrate  that  such  acquisitions  are  the  most 
appropriate  means  for  meeting  its  immediate  needs. 


Further,  gao  recommends  that  the  Secretary  direct  the  Commissioner  to 
report  the  lack  of  effective  capacity  management  as  a  material  control 
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weakness  under  the  Federal  Managers’  Financial  Integrity  Act  until  an 
improved  program  has  been  implemented.  (See  p.  26.) 


Agency  Comments 


In  commenting  on  a  draft  of  this  report,  the  Department  of  Health  and 
Human  Services  agreed  with  gao’s  recommendations  on  the  need  for 
improvements  in  ssa’s  capacity  management  program.  The  Department 
also  commented,  however,  that  it  is  uncertain  whether  ssa's  improved 
capacity  management  program  will  be  fully  implemented  in  time  to  jus¬ 
tify  a  major  computer  acquisition  planned  for  1989  or  1990.  In  that  case, 
SSA  plans  to  develop  the  data  necessary  to  support  such  a  procurement. 
GAO  agrees  that  ssa  should  demonstrate  through  sufficient  data  and 
thorough  analysis  that  such  acquisitions  are  the  most  appropriate  for 
meeting  the  agency’s  immediate  needs.  (See  pp.  26  and  27.) 

The  Department  disagreed  with  gao’s  recommendation  that  it  report  the 
lack  of  effective  capacity  management  as  a  material  control  weakness 
under  the  Federal  Managers’  Financial  Integrity  Act.  gao  believes  that  a 
capacity  management  program  is  an  integral  part  of  effective  computer 
operations.  Since  computer  operations  are  central  to  ssa  fulfilling  its 
mission,  gao  believes  the  inadequacy  of  ssa’s  program  constitutes  a 
material  weakness  under  the  Act.  (See  p.  28.) 


Page  5 


GAO  U.nXC  89  8  SS.\’s  Caparity  Mana4<>in<>nt  Program 


Contents 


Executive  Summary 

2 

Chapter  1 

8 

Introduction 

SSA’s  Computer  Modernization  Plan 

8 

Capacity  Management  Is  an  Integral  Part  of  Computer 
Operations 

9 

Objective,  Scope,  and  Methodology 

11 

Chapter  2 

13 

Improvements  Needed 

Sufficiently  Detailed  Data  Not  Systematically  Collected  or 

14 

in  SSA’s  Capacity 

Analyzed 

Insufficient  Information  Adversely  Affecting  Accuracy  of 

15 

Management  Program 

System  Modeling 

Agency  Comments  and  Our  Evaluation 

19 

Chapter  3 

20 

Independent 

SSA  Evaluated  Its  Capacity  Management  Program 

20 

Assessments  Find 

SSA  Contractors  Evaluated  SSA’s  Capacity  Management 
Program 

22 

Significant 

Efforts  to  Improve  Capacity  Management  Not  Yet 

23 

Improvement  Needed 
in  SSA’s  Capacity 
Management  Program 

Complete 

Agency  Comments  and  Our  Evaluation 

23 

Chapter  4 

25 

Conclusions  and 

Recommendations 

26 

Recommendations 

Agency  Comments  and  Our  Evaluation 

26 

Appendixes  Appendix  I:  Agency  Comments  30 

Appendix  II;  M^or  Contributors  to  This  Report  34 


Abbreviations 

ADP  automated  data  processing 

GAO  General  Accounting  Office 

IMTEC  Information  Management  and  Technology  Division 

SSA  Social  Security  Administration 


Page  6 


GAO/IMTE0894  SSA's  Capacity  Management  Program 


Pub**  7 


GAO/  nVTTEO*"  <*  SSA's  Capacity  ManaBcment  ProBram 


Chapter  1 


Introduction 


Social  Security  Administration  (ss.a)  programs  affect  millions  of  wage 
earners  and  StKiai  Security  beneficiaries.  The  agency's  primary  pro¬ 
grams  are  the  Retirement,  Surv  ivors,  and  Di.sability  Insurance  Program 
and  the  Supplemental  Security  Income.  Aged.  Blind,  and  Disabled 
Program. 

ssa's  computer  operations  in  support  of  these  programs  are  extensive 
because  they  serve  a  large  client  population  and  maintain  records  for 
past,  current,  and  future  beneficiaries.  The  agency  maintains  one  of  the 
largest  automated  collections  of  data  in  the  world.  s.s.a  computer  opera¬ 
tions  process  monthly  benefit  payments  to  over  40  million  pei>ple.  and 
maintain  master  record  information  for  over  300  million  p<>ople.  Total 
annual  benefit  payments  exceed  $230  billion.  The  data  is  contained  in 
files  that  are  accessed  and  updated  when  inquiries  are  made  against 
them,  when  social  security  beneficiaries  make  claims,  or  when  benefi¬ 
ciaries  move  or  change  eligibility  status. 

SSA  spends  hundreds  of  millions  of  dollars  every  year  to  maintain  and 
modernize  its  computer  systems.  For  instance,  ss.a  estimated  in  1987 
that  the  agency  would  spend  approximately  $643  million  from  fiscal 
year  1982  through  fiscal  year  1988  to  modernize  and  maintain  its  com¬ 
puter  operations  and  is  currently  planning  to  spend  another  $  170  mil¬ 
lion  in  fiscal  year  1989. 


To  improve  its  computer  systems,  s.sa  proposed  a  comprehensive  5-year 
Systems  Modernization  Plan  in  1982.  A  primary  emphasis  of  the  plan 
was  to  allow  more  immediate  service  to  clients  while  they  were  in  an  .ssa 
field  office  through  a  new  on-line  system.'  While  s.sa  uses  batch-  and  on¬ 
line  systems  to  serve  clients,  a  goal  of  the  systems  modernization  was  to 
increase  on-line  transactions.  In  order  to  implement  the  new  on-line  sys¬ 
tem,  s,sa: 

•  is  replacing  and  expanding  terminal  and  communications  equipment  in 
field  locations  by  replacing  a  system  of  4,200  terminals  and  the  associ¬ 
ated  older  communications  network''  that  supported  certain  client  ser¬ 
vices.  with  25,530  new  terminals  and  a  new  data  communications 


SSA’s  Computer 
Modernization  Plan 


'  An  on-line  system  forwards  transactions  to  the  computer  and  pr(K-es.s<'S  them  a.s  stnm  as  they  arc 
received 

^A  batch  .system  collects  and  stores  traasaefions  in  batches  for  processing  and  reports  all  e.vceptions 
within  a  batch  after  processing  has  occurred 

"Social  Security  Administration  Data  .Vcquisition  and  Response  System 
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network.  The  increased  number  of  terminals  will  give  field  office  repre¬ 
sentatives  additional  devices  for  accessing  ssa's  master  files. 

•  is  redesigning  and  upgrading  its  beneficiary-related  software  applica¬ 
tions.  The  Claims  Modernization  Project,  which  is  part  of  this  effort, 
involves  modernizing  the  proce.s,ses  for  collecting  initial  claims  data  and 
determining  eligibility  and  benefit  amounts  by  rt'designing  claims  soft¬ 
ware.  As  of  April  1988,  redesigned  software  had  been  installed  to  allow 
field  staff  to  directly  enter  initial  claims  information  through  computer 
terminals  rather  than  requiring  clients  to  fill  out  forms.  Other  claims 
modernization  efforts  under  development  will  allow  field  .staff  to  deter¬ 
mine  client  eligibility  and  compute  benefit  amounts  immediately  through 
the  new  on-line  system.  Cnder  the  current  (batch  oriented)  system,  the 
client  submits  required  information  while  in  the  field  office,  and  must 
wait  until  some  later  date  for  s.s.\  to  make  a  determination  regarding  eli¬ 
gibility  status  and  benefit  amounts. 

•  has  improved  the  data  storage  and  access  methods  of  s.s.\'s  master  com¬ 
puter  files  to  enhance  access  to  the  master  files  by  field  locations. 

•  has  replaced  the  computers  in  ss.\'s  main  computer  center  with  new. 
more  pow'erful  computers  to  support  the  processing  requirements  of  the 
on-line  system. 

ssA  is  currently  installing  the  hardware  and  software  to  implement  its 
on-line  system.  In  June  1987,  ssA  purcha.st'd  and  installed  three  main¬ 
frame  computers  in  the  headquarters  data  center  to  serve  as  pnKessing 
units  for  the  25,530  terminals  being  installed  in  headquarters,  in 
approximately  1340  field  offices,  and  in  seven  processing  centers 
thr(.tughout  the  country.  s.sa  had,  as  of  St'ptember  1988,  installed  23,570 
of  its  planned  25,530  terminals. 

Concurrent  with  the  implementation  of  the  new  on-line  system,  .s.sa  is 
phasing  out  its  older  terminals  and  data  communications  sy.stem.  Once 
the  functions  currently  performed  on  the  older  network  are  imple¬ 
mented  on  s-sa’s  new  data  communications  network,  the  older  network 
will  be  phased  out. 


Capacity  Management 
Is  an  Integral  Part  of 
Computer  Operations 


Capacity  management  programs  provide  a  capability  to  ensure  that 
computer  systems  are  ( 1 )  properly  designed  and  configured  to  give  effi¬ 
cient  performance,  and  (2)  have  sufficient  computer  capacity  for  pre¬ 
sent  and  future  operations.  Federal  Information  Resources  Management 
Regulation  parts  201-19,  201-20,  and  201-30  require  government  agen¬ 
cies  to  conduct  capacity  management  activities  in  planning,  acquiring. 
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and  using  their  computer  resources.  The  regulations  require  that  agen¬ 
cies  do  short-  and  long-range  acquisition  planning,  which  should  include: 

•  analyzing  trends  in  data  processing  work  loads  to  determine  if  and  when 
existing  system  capabilities  will  be  saturated,  and 

•  conducting  a  comprehensive  requirements  analysis,  including  the  pre¬ 
sent  and  projected  work  load  in  terms  of  data  handling  or  transaction 
processing  by  type  and  volume,  to  justify  the  acquisition  of  additional 
computer  capacity. 

The  National  Bureau  of  Standards  offers  work  load  analysis  and  fore¬ 
casting  guidance  to  federal  agencies.  For  example,  the  Bureau's  publica¬ 
tion  on  computer  performance  management  (  Publication  49)  states  that 
automated  data  processing  (.adi>)  managers  must  have  detailed  informa¬ 
tion  concerning  current  computer  work  load  in  order  to  enhance  the  cur¬ 
rent  system  effectively  and  to  constnict  valid  models  with  w'hich  to 
predict  future  resource  requirements.  Standard  industry  practices  also 
suggest  that  organizations  institute  effective  capacity  management  pro¬ 
grams  to  understand  how  resources  can  be  used  to  make  computer  sys¬ 
tems  perform  as  efficiently  as  possible  and  to  plan  ahead  for  future 
needs. 

Capacity  management  includes  performance  management  and  capacity 
planning.  Performance  management  involves  analyzing  the  performance 
of  a  computer  system  to  determine  how  resources  are  currently  utilized 
and  how  such  utilization  can  be  improved.  It  is  the  part  of  capacity  man¬ 
agement  that  deals  with  the  day-to-day  requirements  of  delivering 
acceptable  service  to  users.  Levels  of  service  are  gauged  by  such  meas¬ 
ures  as  response  time,  job  completion  (turnaround)  time,  availability 
and  reliability  of  computer  equipment,  cost  of  operation,  and  accuracy 
of  output.  Specific  activities  consist  of  collecting  and  analyzing  informa¬ 
tion  on  the  performance  of  an  existing  system  and  locating  real  or  poten¬ 
tial  bottlenecks  in  system  performance.  For  example,  if  good 
performance  management  activities  are  in  place,  an  agency  can  deter¬ 
mine  whether  its  data  access  system  is  efficient.  If  tests  and  machine 
monitoring  devices  indicate  that  slow  access  to  stored  data  is  adversely 
affecting  user  response  time,  the  agency  can  make  adjustments,  such  as 
adding  paths  (or  channels)  from  the  computer  to  the  data  storage 
devices  to  enhance  system  performance.  Ongoing  performance  manage¬ 
ment  activities  can  locate  trouble  spots  and  enable  the  agency  to  make 
necessary  adjustments  to  optimize  computer  performance. 
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Capacity  planning  assists  in  forecasting  computer  resource  requirements 
to  ensure  that  capacity  exists  when  needed.  Capacity  planning  supports 
the  procurement  process  by  identifying  and  justifying  system  additions 
and  enhancements  that  will  be  required  both  in  the  near  and  long  term. 
Capacity  planning  activities  use  current  system  performance  data  as  a 
starting  point  to  predict  future  resource  needs.  Modeling  and  pilot  test¬ 
ing  are  two  activities  that  can  be  used  in  capacity  planning  to  provide 
data  on  future  requirements  and  system  perhirmance.  During  the  capac¬ 
ity  planning  process,  the  total  future  work  load  and  required  user  ser¬ 
vice  levels  are  predicted,  resources  required  to  handle  the  work  load  and 
service  levels  are  proposed,  planned  upgrades  are  modeled  and  pilot 
tested,  and  the  ultimate  configuration  is  defined. 


Objective,  Scope,  and 
Methodology 


The  objective  of  our  review  was  to  evaluate  the  effectiveness  of  ss.'t  s 
activities  for  managing  existing  computer  resources  and  planning  for 
future  resource  needs.  To  meet  our  objective,  we; 


•  interviewed  ssa's  managers  for  hardware  and  software  activities  regard¬ 
ing  procedures  used  to  analyze  past  and  current  performance  of  the  on¬ 
line  system, 

•  reviewed  data  collection  documents,  such  as  computer  monitoring 
reports  and  reports  of  modeling  results  and  pilot  test  results  to  deter¬ 
mine  ( 1 )  how  these  activities  have  been  developed  at  ss.^,  and  (2)  the 
typos  of  information  being  collected  at  s,s.\, 

•  reviewed  reports  from  ss.x’s  integration  and  management  contractor  on 
s.s.a's  integration  planning  process  and  ss.x's  integrated  management 
plan. 

•  reviewed  a  report  from  .ssa’s  contractors  on  modeling  activities, 

•  interviewed  s,sa  officials  responsible  for  managing  computer  procure¬ 
ments  and  submitting  budgets  to  evaluate  how  these  functions  were 
integrated  into  the  capacity  management  function, 

•  reviewed  documents  pertaining  to  s.sa's  planned  hardware  procurements 
and  planned  software  implementation  schedule  to  evaluate  whether 
the.se  were  adequately  coordinated  with  capacity  management  activi¬ 
ties,  and 

•  evaluated  reports  on  18  field  offices,  regional  facilities,  processing  cen¬ 
ters,  and  a  teleservice  center,  designated  as  “barometer  offices,"  to  sup¬ 
ply  daily  performance  feedback  from  system  users  to  the  computer 
facilities  at  headquarters. 
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Introduction 


As  criteria  to  determine  the  effectiveness  of  sa^’s  capacity  management 
program,  we  used  the  Federal  Information  Resources  Management  Regu¬ 
lation,  parts  201-16,  201-20,  and  201-30  and  the  National  Bureau  of 
Standards  work  load  analysis  and  forecasting  guidelines  in  Federal 
Information  Processing  Standards  Publications  49  and  75.  We  also 
talked  to  management  at  the  National  Institutes  of  Health,  the  Depart¬ 
ment  of  Justice,  and  a  private  life  insurance  company  to  get  a  broader 
perspective  on  capacity  management  activities  at  other  large  organiza¬ 
tions  that  have  large  computer  facilities. 

Our  audit  work  was  conducted  between  January  1987  and  September 
1988.  Our  work  was  performed  in  accordance  with  generally  accepted 
government  auditing  standards.  We  provided  a  copy  of  a  draft  of  this 
report  to  the  Department  of  Health  and  Human  Services  for  its  review 
and  comment.  The  Department’s  comments  are  included  in  their  entirety 
in  appendix  I  and  are  summarized  at  the  end  of  each  chapter  of  this 
report  when  pertinent. 
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ska’s  initiative  to  modernize  its  computer  operations  and  implement  a 
new  on-line  system  includes  changes  in  many  components  of  the  com¬ 
puter  environment — redesigned  software,  new  data  communications 
and  operating  systems,  different  data  storage  devices,  and  a  new  data 
management  and  access  system.  Federal  regulations  require  that  an 
effective  capacity  management  program  be  in  place  to  provide  the  capa¬ 
bility  to  ensure  that  system  components  operate  together  efficiently, 
and  that  sufficient  computer  capacity  is  available  to  implement  soft¬ 
ware  work  loads  scheduled  to  be  processed  on  the  system. 

We  found  that  ssa  does  not  have  an  effective  capacity  management  pro¬ 
gram.  Although  the  agency  does  some  data  collection  and  analysis,  pilot 
testing,  and  modeling — all  basic  elements  of  a  good  capacity  manage¬ 
ment  program — it  does  not  systematically  collect  and  analyze  suffi¬ 
ciently  detailed  data  on  planned  work  loads  early  enough  in  the 
software  development  process  to  ensure  that  capacity  planning  activi¬ 
ties  can  be  performed  throughout  the  development  cycle.  Because  work 
load  data  is  not  available  when  needed,  the  agency  cannot  model  the 
system  effectively  to  predict  computer  capacity  needs  accurately.  Fur¬ 
ther,  the  agency  has  not  used  pilot  testing  to  (a)  evaluate  how  all  of  the 
system’s  various  components  operate  together,  or  (b)  assess  actual  traf¬ 
fic  patterns  as  the  components  are  installed.  This  inhibits  .ska’s  ability  to 
predict  and  make  adjustments  to  avoid  potential  system  implementation 
problems  and  plan  for  computer  capacity  requirements. 

These  conditions  are  not  new.  In  August  1986,  we  reported  that  ska  was 
acquiring  components  for  its  new  on-line  system  for  field  offices  without 
the  analyses  and  testing  required  by  federal  regulations  to  support  such 
significant  purchases.'  We  were  concerned  that  in  the  absence  of  these 
analyses  and  tests,  s.sa  would  be  unable  to  predict  the  performance  of 
the  fully  deployed  system.  Our  concerns  were  justified  in  that  even 
though  ksa  has  expanded  its  computer  capacity  to  support  the  new  sys¬ 
tem,  it  has  experienced  performance  problems,  such  as  slow  system 
response  times  and  unavailability  of  the  system.  The  agency  now 
believes  that  additional  computer  capacity  is  needed  to  fully  implement 
the  system,  even  though  the  computers  that  were  in.stalled  in  January 
1988  were  intended  to  support  the  entire  system. 


'  .M)!’  .Acquisitions:  SS.A  Should  l.imil  .ADI’  PriH  urcmcnts  I  ntil  P'lirthcr  Testing  is  I’crformt'd.  (GAO 
IMTlX’-86-Jl..Aug.  8.  mSii). 
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Sufficiently  Detailed 
Data  Not 
Systematically 
Collected  or  Analyzed 


Both  performance  management  and  capacity  planning  rely  on  the  collec¬ 
tion  of  detailed  and  accurate  performance  data  on  computer  activities  to 
monitor  current  operations,  such  as  computer  capacity  utilization  and 
average  response  time,  and  to  predict  future  capacity  requin*ments,  s.sa 
does  not  systematically  collect  and  analyze  sufficiently  detailed  capacity 
management  data  on  current  and  future  computer  operations. 

Federal  Information  Resources  Management  Regulation  201-30  (Manage¬ 
ment  of  ADP  Resources)  states  that  agency  computer  acquisitions  and  on¬ 
going  computer  operations  should  be  supported  by  information  on  the 
present  and  projected  work  load  in  terms  of. 

data  handling  or  transaction  processing  by  type  and  volume, 
output  needs  and  associated  telecommunications  support,  and 
a  performance  evaluation  of  the  currently  installed  adp  system(s)  to 
provide  a  baseline  for  evaluation  of  proposed  alternatives  for  meeting 
data  processing  needs. 

In  our  August  1986  report,  we  noted  that  s,sa  was  determining  its  com¬ 
puter  resource  requirements  from  work  load  analysis  for  a  number  of 
software  requirements  that  had  not  been  defined  or  validated.^  As  a 
result,  the  work  load  projections  that  s.sa  performed  produced  results 
that  varied  widely.  For  example,  a  critical  work  load  element  of  the  sys¬ 
tem — transactions  per  terminal  per  hour — had  been  estimated  in  one 
study  to  be  10.8  and  in  another  to  be  36.4.  Such  variances  in  data  make 
predicting  resource  requirements  difficult  because  the  lower  transaction 
rate  per  hour  would  require  considerably  less  computer  capacity  than 
the  higher  transaction  rate.  We  recommended  that  s,sa  not  commit  to 
procuring  all  of  the  on-line  system  computers  and  other  equipment  until 
more  reliable  work  load  data  were  gathered  and  analyzed. 

In  August  1986,  SSA  began  preparing  capacity  baseline  chart  reports  that 
contain  utilization  information  for  various  components  of  the  on-line 
system,  such  as  the  percentage  of  time  a  computer  is  in  use  throughout 
the  day  or  the  hours  of  maximum  computer  utilization.  This  data,  how¬ 
ever,  is  not  sufficiently  detailed  for  use  in  computer  capacity  planning. 
For  example,  although  s,sa  collects  data  on  the  total  computer  resources 
consumed  by  batch  and  on-line  software  applications,  it  does  not  collect 
performance  data  by  individual  applications.  Federal  Information 
Resources  Management  Regulations  require  that  agencies  consider  such 
detailed  information  during  requirements  analysis  because  knowledge  of 
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Insufficient 
Information  Adversely 
Affecting  Accuracy  of 
System  Modeling 


individual  work  load  growth  patterns  helps  to  accurately  predict  future 
computer  capacity  requirements. 

In  addition  to  not  collecting  sufficiently  detailed  data  on  its  software 
applications,  ssa  does  not  have  mechanisms  in  place  to  ensure  that 
appropriate  procedures  for  information  collection  are  followed.  For 
example,  ssa’s  integration  and  management  contractor  said  that  there 
currently  are  several  ssa  forms  associated  with  adp  planning  that  would 
supply  useful  capacity  information.’  However,  the  contractor  noted  that 
these  forms  are  optional  and  ask  very  technical  questions  that  users  and 
requirements  developers  cannot  always  answer.  As  a  result,  these  forms 
are  not  routinely  completed  for  the  Office  of  Systems  Operations  to  use 
in  planning  future  re.source  needs 

The  SSA  Division  Director.  Division  of  Operational  Capacity  Performance 
Management,  and  staff  said  that  implementing  and  enforcing  a  data  col¬ 
lection  program  across  many  different  organizations  in  the  agency  has 
been  difficult  to  coordinate.  They  added  that  (1)  the  current  information 
format  oi  the  baseline  utilization  reports  is  used  mainly  to  give  manage¬ 
ment  an  overview  of  computer  operations  rather  than  for  capacity  plan¬ 
ning  purposes,  (2)  more  detailed  information  is  necessary  to  be  useful 
for  capacity  management  activities,  and  (3)  organizing  the  routine  col¬ 
lection,  retention,  and  reporting  of  data  into  useful  performance  infor¬ 
mation  would  give  responsible  staff  more  time  to  analyze  how  to 
improve  computer  performance.  They  are  currently  evaluating  alterna¬ 
tives  for  implementing  an  improved  data  collection  system. 


Although  .SSA  uses  computer  modeling  as  a  technique  to  project  adp 
needs,  the  lack  of  work  load  information  on  planned  software  applica¬ 
tions  has  prevented  the  agency  from  generating  accurate  results  from  its 
modeling  efforts.  Consequently,  s.sa’s  modeling  projection  of  its  current 
computer  capacity  ranges  from,  in  the  best  case  scenario,  being  suffi¬ 
cient  to  meet  the  agency’s  needs  through  calendar  year  1992,  to,  in  the 
worst  case  scenario,  meeting  the  agency’s  needs  only  through  calendar 
year  1988.  Further,  while  pilot  testing  can  generate  information  for  use 
in  increasing  the  reliability  of  modeling,  .ssa’s  pilot  testing  has  not  been 
u.sed  to  test  all  components  of  its  planned  computer  system.  Conse¬ 
quently,  the  agency’s  pilot  tests  were  not  sufficient  to  predict  the  work 


’ssa  contranod  with  an  integration  and  management  eontraetor  in  IP87  to  prov  ide  information  that 
enlianees  the  eapability  of  senior  management  to  measure  the  agency's  o\  erall  colks  tive  progress  in 
accomplishing  strategic  goals  and  provides  senior  managem('nt  with  a  basis  for  dtx  ision  and  action 
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load  that  w  as  generated  as  ssa  implemented  the  various  elements  of  its 
planned  system. 


Detailed  Information  Not 
Available  for  System 
Modeling 


Computer  modeling  is  a  technique  that  can  be  used  to  meet  Federal 
Information  Resources  Management  Regulation  parts  201-16  and  201-30 
that  require  agencies  to  periodically  determine  their  capability  to  meet 
current  and  projected  adp  needs.  Modeling  is  an  iterative  process  that 
begins  when  the  requirements  for  a  software  application  have  been 
defined  and  becomes  increasingly  more  detailed  as  the  complete  system 
under  development  progresses.  In  order  to  predict  both  total  system  per¬ 
formance  and  total  system  capacity  requirements,  all  of  the  software 
applications  and  their  associated  expected  work  loads  that  will  be 
processed  on  the  system  should  be  included  in  the  model.  For  instance,  if 
a  computer  system  is  scheduled  to  process  three  separate  software 
application  work  loads,  these  should  be  modeled  together  along  with 
simulations  of  the  proposed  computer  operating  system,  telecommunica¬ 
tions  system,  data  base  management  system,  and  other  system  compo¬ 
nents.  Without  such  a  simulation,  it  is  difficult  to  accurately  predict  how 
the  complete  system  will  operate,  or  how  much  computer  capacity  will 
be  required. 


In  February  1988,  the  Deputy  Commissioner,  Office  of  Operations,  and 
staff,  told  us  that  of  the  seven  software  application  work  loads  being 
modeled,  only  one  had  sufficient  actual  data  for  predicting  performance 
characteristics  or  capacity  requirements  with  any  reliability.  While  data 
was  available  on  two  other  software  application  work  loads,  the  officials 
said  the  data  was  less  detailed  and  was  of  less  value  in  predicting  future 
performance  or  capacity  requirements.  The  officials  said  that  they  were 
in  the  data  gathering  stage  for  the  remaining  software  application  work 
loads  scheduled  for  modeling,  and  could  not  predict  the  performance  or 
capacity  requirements  of  them. 


Without  detailed  information  on  these  work  loads,  ssa  has  been  unable 
to  accurately  model  and  predict  capacity  requirements  for  the  system 
work  loads  with  any  degree  of  reliability.  For  example,  the  agency  esti¬ 
mated  as  late  as  May  1987,  that  three  computers,  procured  in  June  1987 
with  installation  completed  in  January  1988,  would  have  sufficient 
capacity  to  process  all  of  the  software  application  work  loads  scheduled 
for  the  on-line  system.  Because  of  delays  and  changes  in  software  design 
specifications,  only  two  of  the  originally  scheduled  work  loads  are  cur¬ 
rently  on-line.  However,  studies  conducted  in  October  1 987  estimated 
that  these  two  software  application  work  loads  alone  could  exceed  the 
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capacity  of  the  three  new  computers,  ssa  said  they  may  have  underesti¬ 
mated  the  computer  capacity  requirements  for  the  total  on-line  system 
by  as  much  as  75  percent. 

In  February  1988,  the  ssa  Director,  Operational  Capacity  Planning  and 
Management,  told  us  that  the  agency  had  projected  a  best,  most  likely, 
and  worst  case  scenario  of  future  computer  resource  needs  for  its  on-line 
system.  The  official  also  said  computer  capacity  estimates  from  model¬ 
ing  activities  varied  according  to  ( 1 )  the  number  of  transactions  per 
claim,  and  (2)  the  version  of  modernized  software  to  be  processed  by  the 
on-line  system.  Because  ssa  did  not  have  firm  information  on  the 
number  of  transactions  per  claim  using  modernized  software,  the  esti¬ 
mated  transactions  used  in  the  model  ranged  from  120  to  180  transac¬ 
tions  per  claim,  with  the  most  likely  projection  being  150  transactions 
per  claim.  Consequently,  the  results  of  the  February  1988  modeling  were 
inconclusive  because  of  the  uncertainly  of  transaction  rates  for  the  ver¬ 
sion  of  software  used  in  the  model.  Using  the  best  case  scenario,  ssa  pre¬ 
dicted  that  its  on-line  computer  capacity  could  be  sufficient  until  at  least 
1992.  However,  the  worst  case  scenario  predictions  were  that  capacity 
would  be  exceeded  late  in  calendar  year  1988,  while  the  most  likely  case 
scenario  predictions  were  that  capacity  would  be  exceeded  in  the  second 
half  of  1989. 

In  September  1988,  ssa  spent  $5.1  million  to  increase  the  capacity  of  the 
computers  purchased  in  June  1987.  ssa  is  presently  preparing  justifica¬ 
tions  for  procuring  additional  computers  in  1989  and  possibly  in  1990. 
The  agency  believes  that  more  capacity  is  necessary  to  process  new  soft¬ 
ware  planned  for  the  on-line  system.  However,  unless  ssa  refines  its 
capacity  forecasting  efforts,  the  agency  will  not  be  able  to  accurately 
predict  capacity  requirements  for  the  new  software. 


Agencies  are  required  to  use  validation  techniques  (such  as  pilot  tests) 
to  reduce  the  risks  that;  ( 1 )  insufficient  capacity  or  degraded  adp  per¬ 
formance  may  adversely  affect  an  agency’s  mission  or  (2)  excessive 
capacity  or  capability  may  be  acquired.  Data  from  pilot  tests  can  be 
used  to  verify  or  refine  modeling  assumptions  by  comparing  these 
assumptions  against  data  accumulated  from  actual  testing  during  the 
pilot. 

The  more  complete  the  pilot  tests,  the  more  useful  the  data  that  it  pro¬ 
duces.  While  SSA  established  a  pilot  test  program  for  the  on-line  system, 
the  program  was  primarily  designed  to  validate  new  or  revised  claims 
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software  to  ensure  that  the  software  was  accurate,  comprehensive,  and 
adaptable  to  field  operations.  In  August  1986,  we  reported  that  .ssa's 
pilot  test  would  not  provide  an  adequate  test  of  the  proposed  system’s 
performance  because  it  was  not  a  comprehensive  test  of  the  proposed 
configuration  of  the  hardware  and  software  applications  under  develop¬ 
ment. ^  At  the  time,  the  Secretary  of  Health  and  Human  Services  said 
that  s,sA  had  sufficient  pilot  experience  to  support  the  procurement  and 
installation  of  its  network  and  terminals  nationwide. 

Also  in  December  1986,  we  reported  that  ssa  had  originally  planned  to 
install  and  test  about  20  terminals  at  each  of  18  pilot  test  field  offices.'' 
However,  because  the  agency  had  difficulties  obtaining  the  planned 
number  of  terminals,  it  installed  and  te.sted  only  three  terminals  at  each 
of  the  18  field  offices.  At  that  time,  we  reported  that  with  this  reduced 
number  of  terminals,  .s.sa  had  not  obtained  sufficient  information  to  pro¬ 
vide  conclusive  results  on  the  claims  modernization  project's  inipact  on 
the  operational  performance  of  the  system,  field  office  positions,  or  ser¬ 
vice  to  the  public.  We  also  noted  that  the  agency’s  test  was  limited 
because  it  evaluated  individual  hardware  and  software  components  in  a 
piecemeal  fashion,  rather  than  providing  for  an  overall  and  thorough 
measure  of  projected  performance  of  the  components  before  they  are 
deployed. 

In  December  1987,  the  s.sa  Deputy  Associate  Commissioner,  Office  of 
Systems  Integration,  told  us  that  field  offices  had  been  experiencing  per¬ 
formance  and  reliability  problems  with  the  new  on-line  system.  The 
problems  included  slow  terminal  response  times,  excessive  periods  when 
the  system  was  inoperable,  and  difficulty  in  gaining  access  to  the  sys¬ 
tem.  The  official  believed  the  problems  were  caused  in  part  by  large 
work  load  increases  caused  by  the  additional  terminals  being  installed  in 
field  offices  nationwide.  In  addition  to  supporting  the  redesigned  soft¬ 
ware,  these  terminals  provided  expanded  on-line  access  to  information 
in  existing  master  files.  The  official  attributed  this  large  increase  in 
activity  to  a  work  load  that  existed  in  the  field  offices  but  was  not  eval¬ 
uated  during  the  pilot  tests  that  were  designed  to  test  the  software  and 
not  the  expanded  on-line  access  capability.  With  the  installation  of  the 
new  terminals  in  the  field  offices,  this  work  load  became  apparent  and 
caused  problems. 


.Acquisitions:  SS.A  Should  Ijmit  .ADI’  I’riH'urcments  I  ntil  Kurthcr  IVslinU  Is  I’l-rfornicd  (G.AO 
I.MTK’-S0-3I.  Aug.  8.  I38(ii 

"’Software  Systems:  SS.A  Kneountering Significant  IH-lavs  In  Its  Claims  Modernization  rrojts  t  lO.AO 
IMt6C-S7-8.  fH-c.  22.  instil  ^  ’ 
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In  its  comments  on  a  draft  of  this  report,  the  Department  of  Health  and 
Human  services  said  that  they  agree  that  ssa  did  not  adequately  pilot 
the  use  of  existing  on-line  access  to  master  files.  The  Department  said 
that  it  was  their  belief  that  since  this  access  was  already  available  for 
several  years,  the  agency  could  predict  with  some  accuracy  the  growth 
that  would  take  place  when  more  terminals,  with  the  same  access  capa¬ 
bilities,  were  installed  in  the  field.  However,  the  significantly  higher  use 
of  this  access  capability  by  field  personnel  who  deal  directly  with  the 
public  was  not  anticipated. 

The  Department  said  that  ssa  did  pilot  its  new  modernized  claims  soft¬ 
ware  extensively.  During  the  pilot  operation,  the  agency  found  addi¬ 
tional  functions  that  would  improve  the  claims  process  for  the  public. 
These  improvements  resulted  in  a  need  for  additional  capacity,  ssa 
agrees  that  allowing  more  users  to  access  existing  software  in  a  new 
.system  should  be  piloted.  The  Department  said  that  the  agency  will  fully 
pilot  test,  when  possible,  the  complete  system  when  major  changes  are 
made.  We  agree  that  pilot  testing  the  complete  environment  when  major 
changes  are  made  would  enable  the  agency  to  predict  its  resource 
requirements  and  the  projected  performance  of  all  hardware  and  soft¬ 
ware  components. 


Pagp  19 


GAO  IMTEC-89-8  SSA's  Capacity  Managempnt  Program 


Chapter  3 _ 

Independent  Assessments  Find  Significant 
Luprovcment  Needed  in  SSA’s  Capacity 
Management  Program 


SSA  has  conducted  its  own  internal  evaluation  of  its  capacity  manage¬ 
ment  program  and  had  two  contractors  perform  independent  assess¬ 
ments  of  the  program.  The  evaluations  have  found  similar  problems  in 
the  agency’s  capacity  management  program  such  as  insufficient  data 
collection,  and  inadequate  capacity  management  activities  to  justify 
procurements.  The  evaluations  have  recommended  corrective  actions. 

Until  the  agency  acts  to  correct  its  capacity  management  program,  it 
will  not  be  able  to  effectively  ensure  that  procurements  of  additional 
computer  resources  are  the  best  means  to  meet  the  agency’s  needs.  The 
inadequacy  of  ssa’s  capacity  management  program  constitutes  a  mate¬ 
rial  weakness  under  the  Federal  Managers’  Financial  Integrity  Act  3 1 
U.S.C.  3512  (b)  and  (c).  This  weakness  is  regarded  as  material  given  the 
fact  that  a  capacity  management  program  is  an  integral  part  of  effective 
computer  operations,  which  is  central  to  ssa  fulfilling  its  mission. 

SSA  has  implemented  some  actions,  such  as  better  defining  the  capacity 
management  program;  however,  agency  officials  have  told  us  that  major 
corrective  actions  will  take  about  1  year  to  implement. 


SSA  Evaluated  Its 
Capacity  Management 
Program 


I 


ssa’s  capacity  planning  and  modeling  team  was  formed  in  May  1987, 
approximately  5  months  after  we  began  this  review,  to  improve  the 
agency’s  capacity  planning  process  and  capacity  modeling  program.  The 
team’s  purpose  was  to  develop  a  series  of  models  with  specific 
timeframes  for  completing  them  and  to  develop  a  capacity  planning  pro¬ 
cess,  present  this  process  to  the  Office  of  Systems  Operations  manage¬ 
ment  for  their  modification  and  approval,  and  institutionalize  the 
approved  capacity  planning  process. 

In  October  1987,  the  team  pro]:x)sed  a  capacity  management  program 
that  would  be  the  responsibility  of  the  Office  of  Systems  Operations. 

The  responsibilities  of  the  Office  of  Systems  Operations  include; 


•  directing  the  development,  implementation,  and  maintenance  of  com¬ 
puter  operating  systems; 

•  directing  and  controlling  all  activities  relating  to  the  development, 
implementation,  and  maintenance  of  software; 

•  formulating  an  officewide  plan  for  the  Office  of  Systems  Operations  and 
assigning  responsibility  among  major  Office  components  for  various 
parts  of  the  plan;  and 

•  providing  the  analysis  and  testing  necessary  to  develop  and  maintain 
current,  new,  and  redesigned  systems. 
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In  its  proposal,  the  capacity  planning  and  modeling  team  identified  four 
interdependent  processes  that  constitute  capacity  management:  capacity 
planning,  performance  management,  procurement  management,  and 
resource  -accounting.'  Also,  the  team  identified  problems  and  solutions  in 
each  of  these  areas  and  implications  for  the  Office  of  System  Operations 
if  solutions  are  to  be  implemented.  The  team  said  that  many  of  the  prob¬ 
lems  they  identified  were  a  result  of  responsibilities  being  fragmented 
and  the  lack  of  formal  procedures.  For  instance,  the  team  found  that  the 
existing  capacity  planning  process  was  fragmented  across  several  orga¬ 
nizations,  and  was  not  capable  of  producing  sufficiently  valid  work  load 
projections. 

In  terms  of  performance  management,  the  team's  report  said  that  there 
were  difficulties  in  obtaining  current  performance  data,  and  that  exces¬ 
sive  time  was  spent  gathering  and  maintaining  system  data  rather  than 
analyzing  performance  data.  The  report  also  cited  problems  in  the  pro¬ 
curement  area  including  ( 1 )  the  ad-hoc  character  of  many  ssa  procure¬ 
ments  to  satisfy  immediate  needs  rather  than  long-range,  well-planned 
projects,  (2)  the  difficulty  experienced  in  developing  accurate  and 
dependable  requirements  statements,  and  (3)  the  difficulty  in  proposing 
state-of-the-art  technical  solutions  to  satisfy  requirements. 

Many  of  the  solutions  the  team  proposed  to  address  the  capacity  man¬ 
agement  problems  require  the  consolidation  and  reorganization  of 
responsibilities  within  the  Office  of  System  Operations,  and  the  imple¬ 
mentation  of  formal  procedures  to  better  manage  and  procure  computer 
resources.  For  instance,  to  alleviate  problems  in  the  procurement  area 
the  team  proposed  that  ssa: 

improve  the  capacity  planning  process  to  improve  the  quality  of  the 
requirements  analysis  and  the  timeliness  of  developmental  efforts, 
concentrate  the  responsibility  for  procurements  in  order  to  focus  the 
required  expertise  and  managerial  responsibility,  and 
develop  and  publish  updated  agency  procurement  procedures  to  clarify 
procurement  documentation  requirements  and  the  approval  process. 

The  team  also  proposed  organizational  changes  to  support  capacity 
management  implementation,  including  the  creation  of  a  procurement 
review  staff  within  the  office  of  the  Associate  Commissioner,  and  the 


'Rt“i<)urc'c  accounting  is  a  means  of  distributing  ADP  costs  to  users  It  is  used  in  many  organizations 
and  is  sometimes  considered  a  part  of  capat  ity  management  because'  it  makes  users  aware  of  .ADP 
resource  utilization  and  the  costs  of  meeting  their  needs 
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formation  of  a  capacity  planning  branch  responsible  for  the  analysis  of 
capacity  requirements,  development  of  computer  configurations,  and 
publication  of  tactical  capacity  plans. 


Two  contractors  have  performed  indepemdent  assessments  of  s.sa’s 
capacity  management  program,  one  as  part  of  a  larger  study,  the  other 
specifically  relating  to  s,sa’s  capacity  management  process.  In  a  March 
1988  study  of  s,sa’s  current  integration  planning  process,  s.sa's  integra¬ 
tion  and  management  contractor  said  that  hardware  procurements  can¬ 
not  be  initiated  until  the  requirements  definition  and  analysis  .stage  of 
the  software  development  life  cycle  is  near  completion.  The  contractor 
said  that  this  W’as  necessary  in  order  to  adhere  to  federal  procurement 
regulations  requiring  detailed  documentation  including  functional 
requirements,  work  load  projections,  analysis  of  alternatives,  and  cost 
benefit  analyses.  Although  mechanisms  in  s.sa‘s  Software  Knginevring 
Technology  Manual  provide  procedures  for  the  Office  of  System  Opera¬ 
tions’  involvement  in  software  project  planning,  in  actuality,  this  dcK's 
not  routinely  happen.  As  a  result,  the  Office  of  System  Operations  often 
made  a  "bc'.st  guess”  as  to  how  much  capacity,  storage,  and  networking 
would  be  needed.  The  contractor  added  that  recently,  the  Office  of  Stra¬ 
tegic  Operations  has  taken  a  more  proac  tive  role  in  requc'sting  and 
obtaining  information  needed  for  capacity  planning.  However,  the  con¬ 
tractor  said  that  the  historical  lack  of  intc'gration  betwc'en  software  and 
hardware  projects  had  contributed  to  current  capacity  problems  and,  as 
a  result,  implementation  of  some  new  software  will  be  delayc'd. 

The  results  of  the  review  that  spec  ifically  relatc‘d  to  ,s.sa's  c  apac  ity  mc»d- 
eling  and  planning  program  were  reported  in  .Inly  1988.  The  review 
fcnind  that  ssa’s  capacity  management  process  as  it  now  stands  is  pro¬ 
ducing  valuable  and  directly  useful  information.  For  example,  the  pro¬ 
cess  identified  ways  to  reduce  the  demands  on  computer  capac  ity  during 
times  when  the  computers  are  heavily  used  by  switching  batch  activity 
from  these  prime  periods  to  times  when  there  is  h'ss  demand  on  com¬ 
puter  capacity.  However,  the  review  also  reportc'd  areas  that  “nc'c'd 
improvement"  or  “need  significant  improvement."  Some  of  the  findings 
of  this  independent  a.ssessment  include: 

•  Agenc  y  prenodures  for  modeling  current  work  loads  were'  in  nc'C'd  of  sig¬ 
nificant  improvement.  For  instance,  the  contractor  identified  eight  crite¬ 
ria  for  success  in  modeling,  including  ( 1 )  detailc'd  levels  of  work  load. 

(’2)  clearly  defined  scenarios,  and  (3)  analysis  of  performance  bottle- 
nc'cks.  The  c’ontractor  found  that  none  of  the  criteria  had  bc'c'ii  met  in 
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modeling  major  current  work  loads,  and  that  no  models  of  the  major 
work  loads  existed  to  support  the  interim  iipgreie  procurement  con¬ 
ducted  by  .S.SA  in  September  1988. 

In  several  cases  ssa’s  projected  capacity  requirements  for  the  agency's 
on-line  work  loads  are  not  justified  with  any  degree  of  precision.  For 
instance,  projections  f(»r  the  future  capacity  requirements  for  a  major 
on-line  work  load  had  been  characterized  as  a  “complete  guess  "  by  the 
S.SA  capacity  management  staff.  The  contractor  said  that  given  the 
jxttential  cost  of  the  recommended  upgrade  and  the  time  available  to 
analyzi'  tiu'  work  load,  a  more  refined  estimate  would  be  ajipropriate. 

On  the  basis  of  the  review ,  the  contractor  concluded  that  the  current 
o-year  forecast  for  s.sa  computer  capacity  might  not  bc'  accurate.  The 
contractor  said  that  (lithe  forecast  rc'quires  additional  refinement  by 
s.s.\  pi'ior  to  the  forc'cast  being  suitable  for  supiK)rting  a  major  procure¬ 
ment  such  as  the  forthcoming  1990  proctirement,  (2)  s.sa  capacity  man¬ 
agement  personnel  are  awari*  of  the  potential  inaccuracic's  in  the 
forc'cast  and  are  actively  working  to  refine  the  forecast,  and  (3)  all  indi¬ 
cations  are  that  subsc'quent  refinemetUs  will  provide  for  comprehensive 
a.ssessments  of  computer  upgrade  irquirements. 


In  .luly  1988.  we  spoke  to  the  Deputy  .Assistant  (Commissioner  for  Sys¬ 
tem  OiH'rations.  the  Semior  Advisor  to  the  IX'puty  Commissioner  for 
()|x*rations,  and  the  Dirc'c  tor.  Division  of  Oixuations,  Capacity  Planning. 
They  told  us  that  s.s.\  is  developing  systems  and  procedures  to  ensure 
that  capacity  planning  issues  arc'  acidrc's.sc'd  by  all  organizational  comix)- 
nents  involved  in  computer  ojx-rations  at  the  appropriate  time.  They 
also  said  that  s.s.\  is  Uxiking  at  the  findings  and  rc'commendations  of  the 
capacity  managc*ment  consultant  and  developing  action  plans  to  make 
the  rc'commendt'd  improvemc'iits  For  example,  ssa  has  followc'd  the 
ca{)acity  management  contractor's  rc'commendation  that  s.sa  c'stablish 
user  service  agrc'emenls  for  its  on-line  system  They  said  th.at  s.sa  should 
be  able  to  complete  other  rc'commc'nded  improvc'inents  in  about  1  yc'ar 


In  its  comments  on  a  draft  of  this  report,  the  fX'partment  of  Health  and 
Human  S<*rvicc*s  said  that  s,s,\  rc'cently  had  a  detailc'd  study  of  its  capac¬ 
ity  management  program  jx'rformed  by  an  exjx'rt  in  the  field  of  capacity 
planning.  The  Dc'partment  commented  that  although  the  study  gave  the 
agency  rea.sonably  high  marks,  it  al.so  made  a  number  c'f  .spc'cific  recom¬ 
mendations  for  improvement  that  were  sound  and  for  which  the  Depart¬ 
ment  has  di'veloix'd  a  comprehensive  plan  of  action  to  c'lisure 
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implementation.  The  Department  also  commented  that  some  of  these 
recommendations  have  already  been  completed  and  the  balance  are 
scheduled  for  completion  by  the  end  of  1P89.  By  that  time,  the  Depart¬ 
ment  predicts  that  the  agency  will  be  in  a  position  where  its  capacity 
management  program  is  routinely  providing  the  kind  of  informaticii 
needed  to.  among  other  things,  justify  the  acquisition  of  necessary  com¬ 
puter  resources.  In  this  chapter,  we  recognize  the  efforts  that  ss.\  is  mak¬ 
ing  to  improve  its  capacity  management  program. 
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ssA  is  at  a  critical  stage  in  its  computer  modernization  effort.  It  is  imple¬ 
menting  a  large,  on-line  system  that  involves  increasing  the  number  of 
ter  minals  at  field  offices,  modernizing  the  data  network,  and  providing 
more  on-line  software  applications.  As  the  system  is  installed  and  imple¬ 
mented,  SSA  field  staff  will  become  increasingly  dependent  on  the  sys¬ 
tem’s  reliability  in  carrying  out  the  day-to-day  mission  of  the  agency. 

Currently,  ssa  does  not  have  an  adequate  capacity  management  program 
to  ensure  that  its  computer  resources  will  be  used  effectively.  Perform¬ 
ance  data  on  current  and  future  software  work  loads  is  not  collected  in 
sufficient  detail  to  be  of  use  in  capacity  management  activities,  such  as 
modeling,  to  predict  resource  needs  of  individual  applications  and  the 
expected  performance  of  the  total  system.  In  addition,  while  pilot  test¬ 
ing  has  been  used  to  test  and  validate  software,  it  has  not  been  used  to 
gather  important  capacity  planning  information  on  all  components  of 
the  on-line  system  or  to  increase  the  reliability  of  the  modeling  process. 
Finally,  although  ssa  says  it  does  not  have  sufficient  computer  capacity 
to  process  additional  software  work  loads,  both  we  and  ssa's  contractors 
believe  that  ssa  has  to  supply  more  specific  analysis  to  justify  procure¬ 
ments  of  additional  hardware. 

In  past  reports,  we  raised  concerns  about  the  potential  consequences  of 
not  effectively  pilot  testing  all  operational  aspects  of  the  new  system 
and  recommended  that  ssa  not  proceed  with  planned  acquisitions  until  it 
had  better  information  to  determine  its  needs.  In  response  to  those 
reports,  the  Secretary  of  Health  and  Human  Services  said  that  ssa  had 
sufficient  pilot  experience  to  support  the  procurement  and  installation 
of  its  network  and  terminals  nationwide  and  proceeded  with  the  acquisi¬ 
tion  of  terminals  and  computers  that  ssa  thought  would  be  adequate  to 
support  the  total  on-line  system.  Now.  although  ssa  estimated  in  1987 
that  the  agency  would  spend  approximately  $643  million  dollars 
through  fiscal  year  1988  to  modernize  and  maintain  its  computer  opera¬ 
tions,  it  has  experienced  slow  response  times  and  periods  when  the  sys¬ 
tem  was  inoperable.  Further,  ssa  now  believes  that  it  needs  additional 
computer  capacity  to  support  the  on-line  system,  although  it  has  not 
adequately  modeled  additional  software  applications  to  accurately  pre¬ 
dict  its  needs. 

SSA  officials  agree  that  capacity  management  activities  have  been  inade¬ 
quate  in  the  past  and  are  trying  to  improve  performance  in  the  agency's 
program.  They  are  currently  reviewing  the  capacity  planning  and  mod¬ 
eling  team’s  proposals  and  their  contractors’  reports  for  improving 
capacity  management,  ssa  has  a  tentative  schedule  of  1  year  to  improve 
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its  capacity  management  program.  However,  despite  acknowledging 
weaknesses  in  its  capacity  management  program,  the  agency  is  planning 
to  procure  additional  computers  because  it  believes  that  it  will  need 
more  capacity  to  implement  additional  on-line  software.  I'ntil  ssa  imple¬ 
ments  a  better  capacity  management  program  it  will  not  be  able  to 
ensure  that  the  planned  procurements  are  the  most  appropriate  actions. 
We  also  believe  that  the  lack  of  an  adequate  capacity  management  pro¬ 
gram  is  a  material  w'eakness  under  the  federal  Managers'  Financial 
Integrity  Act  31  H.S.C.  3512  (b)  and  (c). 

Unless  SSA  improves  its  computer  capacity  management  program,  it  will 
not  be  able  to  ensure  that  it  is  managing  its  computer  resources  effi¬ 
ciently.  In  addition,  without  a  better  capacity  management  program,  .s.sa 
will  not  be  able  to  ensure  that,  in  the  future,  it  procures  the  right  equip¬ 
ment  at  the  right  time  and  in  the  right  amounts  to  support  the  mission 
effectively. 


Recommendations 


GAO  recommends  that  the  Secretary  of  Health  and  Human  Services 
direct  the  Commissioner  of  Social  Security  to. 

ensure  that  s.sa  improves  its  capacity  management  program  as  currently 
scheduled  and  uses  this  program  to  justify  future  acquisitions, 
avoid  executing  major  computer  acquisitions  until  an  effective  capacity 
management  program  is  in  place,  and 

in  the  event  that  s.sa  believes  that  computer  resources  must  be  acquired 
to  support  day-to-day  operations  before  an  effective  capacity  manage¬ 
ment  program  is  in  place,  it  should  demonstrate  through  sufficient  data 
and  thorough  analysis,  that  such  acquisitions  are  the  most  appropriate 
means  for  meeting  its  immediate  needs. 

Further,  gao  recommends  that  the  Secretary  direct  the  Commissioner  to 
report  the  lack  of  effective  capacity  management  as  a  material  control 
weakness  under  the  Federal  Managers'  Financial  Integrity  Act  until  an 
improved  program  has  been  implemented. 


Agency  Comments  and 
Our  Evaluation 


We  requested  comments  on  a  draft  of  this  report  from  the  Department 
of  Health  and  Human  Services.  The  Department  agreed  that  s.sa's  capac¬ 
ity  management  program  was  in  need  of  improvements.  However,  the 
Department  believes  that  the  report  does  not  reflect  some  of  the  changes 
that  the  agency  has  made  in  its  capacity  management  program  since 
1987  and  that  the  agency's  program  is  in  better  condition  than  our 
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assessment  would  indicate.  We  do  refer  in  the  report  to  improvements  in 
program  areas  made  through  September  1988  where  the  agency  had 
documented  these  efforts  for  us.  However,  our  audit  work  since  1987, 
along  with  a  consultant’s  report  dated  July  1988,  demonstrates  that  our 
report  accurately  reflects  the  status  of  ssa’s  capacity  management  pro¬ 
gram  as  of  September  1988.  (see  pp.  22-24). 

The  Department  agrees  with  our  recomrriendation  that  ssa  should  ensure 
improvements  are  made  to  its  capacity  management  program  as  cur¬ 
rently  scheduled  and  use  this  program  to  justify  future  acquisitions.  The 
Department  stated  that  the  agency  is  scheduled  to  implement  all  of  the 
recommended  improvements  by  the  end  of  1989. 

The  Department  also  agrees  with  our  recommendation  that  ssa  should 
avoid  executing  m^or  computer  acquisitions  until  an  effective  capacity 
management  program  is  in  place.  However,  the  Department  does  not 
believe  ssa  should  avoid  acquisitions  until  an  effective  program  is  in 
place  if  such  action  would  put  the  agency’s  mission  at  undue  risk.  The 
Department  states  that  current  plans  call  for  executing  a  major  com¬ 
puter  acquisition  by  the  end  of  1989  or  the  beginning  of  1990.  ssa 
believes  that  this  mjyor  procurement  should  be  executed  even  if  all  of 
ssa’s  capacity  management  improvements  have  not  been  made,  as  long 
as  the  agency  has  provided  sufficient  data  to  support  the  procurement. 
The  Department  believes  it  is  important  to  recognize  that  an  organiza¬ 
tion  can  develop  the  data  necessary  to  support  a  procurement  action 
without  necessarily  having  implemented  every  recommendation  made 
on  its  capacity  management  program. 

Our  recommendation  that  ssa  avoid  executing  major  computer  acquisi¬ 
tions  was  not  intended  to  put  the  agency’s  mission  at  risk.  Rather,  it  is 
intended  to  caution  the  agency  from  proceeding  with  major  procure¬ 
ments  until  an  effective  capacity  management  program  is  in  place  to 
support  the  nature  and  extent  of  planned  computer  capacity  acquisi¬ 
tions.  Also,  we  acknowledge  that  if  the  agency  believes  that  computer 
resources  must  be  acquired  to  support  day-to-day  operations  before  an 
effective  capacity  management  program  is  in  place,  it  should  demon¬ 
strate  through  sufficient  data  and  thorough  analysis  that  such  acquisi¬ 
tions  are  the  most  appropriate  for  meeting  its  immediate  needs.  We 
believe  that  the  Department’s  response  to  this  subsequent  recommenda¬ 
tion — that  appropriate  justification  be  developed  by  ssa  before  it 
acquires  additional  computer  capacity  in  the  near  future — addresses 
our  concern  and  will  help  lay  the  foundation  for  an  improved  and  effec¬ 
tive  capacity  management  program  at  ssa. 
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The  Department  of  Health  and  Human  Services  does  not  agree  with  our 
final  recommendation  that  the  Department  report  the  lack  of  effective 
capacity  management  as  a  material  control  weakness  under  the  Federal 
Managers’  Financial  Integrity  Act  until  an  improved  program  has  been 
implemented.  The  Department  comments  that  it  sees  no  basis  for  this 
recommendation  using  the  established  Office  of  Management  and 
Budget  guidelines  for  the  Federal  Managers’  Financial  Integrity  Act  and 
our  “principles  and  standards”  for  defining  material  weaknesses.  The 
Department  recommended  that  this  last  recommendation  be  deleted 
from  the  final  report. 

We  have  reviewed  the  August  1988  Office  of  Management  and  Budget 
guidelines  for  the  Federal  Managers’  Financial  Integrity  Act  and  believe 
that  they  serve  as  a  basis  for  this  recommendation.  Specifically  in  its 
guidance,  the  Office  of  Management  and  Budget  states  that  for  purposes 
of  determining  what  constitutes  a  material  weakness  in  internal  control 
systems  (Section  2)  the  criteria  set  forth  in  Office  of  Management  and 
Budget  Circular  A- 123  shall  be  used.  The  criteria  provide  for  reporting 
weaknesses  that  among  other  things,  “significantly  impair  the  fulfill¬ 
ment  of  an  agency  or  component’s  mission.”  We  believe  that  ssa’s  lack  of 
an  adequate  capacity  management  program  could  significantly  impair 
the  agency’s  ability  to  efficiently  fulfill  its  mission.  We  have  added  clari¬ 
fication  of  this  point  to  the  introduction  to  chapter  3. 
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DEPARTMENT  OF  HEALTH  &  HUMAN  SERVICES 


Office  of  Inspector  General 


Washington.  0  C  20201 


DEC  z  m 


Mr.  Ralph  V.  Carlone 
Division  Director 
U.S.  General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Carlone: 

Enclosed  are  the  Department’s  comments  on  your  draft  report, 
"Computer  Operations:  Improvements  Needed  in  Social  Security's 
Capacity  Management  Program."  The  enclosed  comments  represent 
the  tentative  position  of  the  Department  and  are  subject  to 
reevaluation  when  the  final  version  of  this  report  is  received. 

The  Department  appreciates  the  opportunity  to  comment  on  this 
draft  report  before  its  publication. 

Sincerely  yours. 


Richard  P.  Kusserow 
Inspector  General 


Enclosure 
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COMMENTS  OF  THE  DEPARTMENT  OF  HEALTH  AND  HUMAN  SERVICES  ON  THE 
GENERAL  ACCOUNTING  OFFICE'S  DRAFT  REPORT,  "COMPUTER  OPERATIONS: 
IMPROVEMENTS  NEEDED  IN  SOCIAL  SECURITY'S  CAPACITY  MANAGEMENT 
PROGRAM" 


General 

We  believe  that  this  report  presents  a  fair  portrayal  of  the 
status  of  the  Social  Security  Administration's  (SSA)  capacity 
management  program  at  the  time  when  most  of  the  General 
Accounting  Office  (GAO)  auditing  was  performed,  which  was  the 
latter  part  of  calendar  year  1967.  After  that  time,  GAO  also 
reviewed  the  reports  on  capacity  management  that  were  prepared  by 
two  contractors  under  contract  to  SSA  and  interviewed  some  agency 
officials  in  mld-1988.  But  we  believe  that  SSA  has  made  some 
improvements  in  its  capacity  management  program  during  1988  that 
GAO  is  not  aware  of  and,  therefore,  are  not  reflected  in  its 
report.  SSA's  capacity  management  program  is  in  better  condition 
than  the  GAO  assessment  would  indicate.  Nonetheless,  we  agree 
with  GAO  that  improvements  have  to  be  made. 

SSA  recently  had  a  detailed  study  of  its  capacity  management 
program  performed  by  International  Systems  Services  Corporation 
(ISS),  an  expert  in  the  field  of  capacity  planning.  While  this 
study  gave  SSA  reasonably  high  marks,  it  also  made  a  number  of 
specific  recommendations  for  improvement.  We  agree  that  the  ISS 
recommendations  are  sound  and  we  have  developed  a  comprehensive 
plan  of  action  to  ensure  that  they  are  implemented.  Some  of 
these  recommendations  have  already  been  completed  and  the  balance 
are  scheduled  for  completion  by  the  end  of  1989.  The  ISS  study 
recognized  that  SSA  was  already  aware  of  most  of  the  needed 
improvements  and  in  many  cases  had  already  begun  corrective 
action.  By  the  end  of  1989  SSA  will  be  in  a  position  where  its 
capacity  management  program  is  routinely  providing  the  kind  of 
information  needed  to,  among  other  things,  justify  the 
acquisition  of  the  necessary  computer  resources. 

GAO  Recommendation 

Ensure  that  SSA  improves  its  capacity  management  program  as 
currently  scheduled  and  uses  this  program  to  justify  future 
acquisitions. 

Department  of  Health  and  Human  Services  Comment 

We  agree  with  this  recommendation  and,  as  indicated  above,  SSA  is 
scheduled  to  implement  all  of  the  recommended  improvements  by  the 
end  of  1989.  Many  improvements  have  already  been  made. 
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GAO  Recommendation 

Avoid  executing  major  computer  acquisitions  until  an  effective 
capacity  management  program  Is  In  place. 


Department  Comment 

We  agree  with  this  recommendation  as  long  as  It  does  not  put 
SSA's  mission  at  risk.  Our  current  plans  call  for  executing  a 
major  computer  acquisition  by  the  end  of  1989  or  the  beginning  of 
1990.  We  believe  that  we  should  execute  this  procurement  even  if 
all  of  SSA's  capacity  management  Improvements  have  not  been  made 
by  that  time,  as  long  as  SSA  has  provided  sufficient  data  to 
support  the  procurement.  We  think  It  Is  Important  to  recognize 
that  an  organization  can  develop  the  data  necessary  to  support  a 
procurement  action  without  necessarily  having  Implemented  every 
recommendation  made  about  Its  capacity  management  program. 

GAO  Recommendation 

If  SSA  believes  that  computer  resources  must  be  acquired  to 
support  day-to-day  operations  before  an  effective  capacity 
management  program  Is  In  place.  It  should  demonstrate  through 
sufficient  data  and  thorough  analysis  that  such  acquisitions  are 
the  most  appropriate  means  for  meeting  Its  immediate  needs. 

Department  Comment 

We  agree  with  this  recommendation.  GAO  has  correctly  recognized 
that  SSA  should  not  be  precluded  from  acquiring  any  necessary 
computer  capacity  until  we  have  completed  all  of  our 
Improvements.  Current  Indications  are  that  we  will  have  to 
acquire  some  additional  capacity  In  the  near  future,  but  we  will 
not  proceed  until  we  have  developed  an  appropriate  justification. 

GAO  Recommendation 

Report  the  lack  of  effective  capacity  management  as  a  material 
control  weakness  under  the  Federal  Managers'  Financial  Integrity 
Act  (FMFIA)  until  an  Improved  program  has  been  Implemented. 

Department  Comment 

We  do  not  concur  with  this  recommendation.  Using  the  established 
C^IB  guidelines  for  FMFIA  and  GAO's  "principles  and  standards"  for 
defining  material  weaknesses,  we  see  no  basis  for  this 
recommendation  and  recommend  It  be  deleted  from  the  final  report. 
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Other  Matters 

Regarding  the  appropriateness  of  pilot  testing,  we  agree  that  SSA 
did  not  adequately  pilot  the  use  of  existing  online  access  to 
master  files.  It  was  our  belief  that  since  this  access  was 
already  available  for  several  years  that  we  could  predict  with 
some  accuracy  the  growth  that  would  take  place  when  more 
terminals  having  the  same  access  capabilities  were  installed  in 
the  field.  What  was  not  anticipated  was  the  significantly  higher 
use  of  this  access  capability.  Once  our  field  personnel  that 
deal  directly  with  the  public  had  access  to  information  and  could 
answer  the  public's  questions  immediately,  more  information 
accessing  was  generated  to  answer  such  questions. 

SSA  did  pilot  its  new  modernized  claims  software  extensively. 
During  this  pilot  operation  SSA  found  additional  functions  that 
would  Improve  the  claims  taking  process  for  the  public.  These 
Improvements  resulted  in  a  need  for  additional  capacity. 

SSA  agrees  that  allowing  more  users  to  access  existing  software 
in  a  new  environment  should  be  piloted  and  we  will  fully  pilot 
test,  when  possible,  the  complete  environment  when  major  changes 
are  made. 
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Management  and 
Technology  Division 
Washington,  D.C. 
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Melroy  D.  Quasney,  Associate  Director,  (202)  275-4659 
Ronald  Maccaroni,  Deputy  Associate  Director 
Anthony  Cicco,  Jr.,  Group  Director 
Margaret  Goodman,  Evaluator-in-Charge 
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